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ABSTRACT 


The  running  state  of  visual  cognition  cannot  be  told  from  the  running  state  of  eye  motion.  Even  the 
simplest  and  most  commonly  accepted  beliefs  fall  apart  when  carefully  studied.  Visual  attention  is 
not  confined  to  the  center  of  gaze  but  can  wander  about  over  the  visual  field  and  become  narrow 
or  wide  depending  on  the  visual  task  to  be  performed. 

We  showed  this  on  seven  subjects  who  were  tested  tachistoscopically  for  form  resolution  and 
identification  simultaneously  at  the  center  of  gaze  and  at  the  near  peripheral  field  of  vision. 

Binocular  eye  motion  as  if  tracking  along  the  boundary  of  an  object  can  occur  without  any  visual 
cognition  that  there  is  such  an  object  in  the  visual  field.  We  showed  this  on  eight  subjects  who 
were  exposed  to  random  dot  stereograms  and  monitored  with  a  state-of-the-art  binocular  eye 
tracker. 

Our  data  led  us  to  believe  that  measures  of  eye  position  and  motion  are  fallible  indices  of  visual 
cognition.  Nevertheless,  as  in  medicine,  signs  of  disorder  may  be  inexplicable  yet  reliable.  Our 
study  leads  us  to  believe  that  tests  can  be  designed  for  cognitive  fatigue  and  overload  using 
transient  and  well-defined  visual  tasks  imposed  on  the  visual  field. 

INTRODUCTION 

Eyes  do  not  necessarily  move  to  where  attention  is  directed,  nor  does  attention  necessarily  center 
on  where  the  eyes  are  directed.  Much  information  in  the  visual  field  can  be  processed  cognitively 
with  but  little  eye  movement.  Furthermore,  in  earlier  studies  we  have  shown  that  the  strategies  of 
that  visual  process  are  learned  and  are  task  specific,  so  that  there  are  many  such  strategies  and  a 
viewer  can  switch  between  them  in  ordinary  vision  depending  upon  his  purpose.  We  can  qualify 
some  of  these  strategies  by  our  tests.  We  have  shown  that  they  can  be  learned  and  that  switching 
between  strategies  can  be  learned. 

The  basic  point  is  that  there  is  no  way  of  telling  from  the  position  or  motion  of  the  eyes  the  visual 
strategy  used  or  the  state  of  visual  cognition.  The  field  of  attentive  and  discriminatory  vision  can 
widen  or  constrict  and  objects  can  assume  distinct  individual  form  or  be  much  degraded  in  their 
visual  aspects  without  any  external  signs  that  this  is  occurring. 


Accordingly,  inferring  the  laws  of  vision,  visual  attention  and  visual  interpretation  from  the  external 
action  of  the  eyes  is  not  a  fruitful  scientific  pursuit. 

However  there  is  a  serious  practical  problem  where  such  study  may  be  implied,  and  that  has  to  do 
with  the  onset  of  visual  fatigue,  cognitive  overload,  or  interpretive  block.  The  subjective  symptoms 
are  not  so  noticeable  nor  are  the  external  signs.  But  in  many  occupations  the  access  of  blunted 
visual  cognition  can  have  disastrous  consequences. 

Our  experiments  suggest  that  no  external  measure  of  the  running  mechanical  activity  of  the  eyes 
informs  us  of  the  running  state  of  visual  perception  and  cognition.  However,  measured 
performance  of  transient  well-defined  tasks  in  visual  perception  gives  consistent  findings  of 
diagnostic  value.  This  suggests  the  design  of  a  new  kind  of  test  for  detecting  visual  fatigue  and 
cognitive  overload. 

The  test  does  not  interfere  with  ongoing  visually  directed  operation,  but  only  checks  occasionally 
the  capacity  to  perform  a  transient  well-defined  visual  task.  The  design  and  instrumentation  of 
such  a  method,  and  testing  it  for  usefulness  compliments  our  ongoing  research. 

Tracking  eye  movement  is  familiar  enough  as  an  instrumental  problem.  Binocular  two-dimension 
tracking  and  its  broad  band  recording  for  detecting  rapid  fine  saccades  as  well  as  courser  ones  are 
well  in  hand.  Our  innovation  with  regard  to  this  newly  refined  measure  has  been  to  show  that  there 
is  no  intelligible  coupling  between  binocular  action  and  conscious  perception  or  cognition.  There 
may  be  a  general  correlation  in  some  crude  average  way,  but  certainly  not  on  careful  study  of  task 
performance.  According  to  the  current  data,  no  casual  relationship  can  be  established  such  that 
we  can  rely  on  ocular  signs  to  indicate  symptoms  of  visual  cognitive  overload  or  impairment. 

The  case  is  different  for  our  method  of  testing  one  feature  of  visual  cognition,  the  FRF  (form¬ 
resolving  field).  Since  we  know  from  eye-tracking  the  position  of  the  center  of  gaze,  the  question  is 
the  position  and  width  of  the  FRF  in  the  visual  field  relative  to  the  center  of  gaze.  It  is  that  solid 
angle  in  which  pairs  of  forms,  such  as  image  objects,  icons,  letters  and  their  changes  can  be 
recognized.  Both  the  width  and  position  of  this  field  with  respect  to  the  center  of  gaze  vary 
depending  on  the  state  of  the  individual,  the  cognitive  task  to  be  performed  and  the  individual 
strategy  of  performance. 
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The  method  has  high  diagnostic  value  in  detecting  dyslexia,  for  example.  It  consists  of 
tachistoscopically  presenting  two  forms  against  a  blank  background,  one  at  the  center  of  gaze,  the 
other  at  some  randomly  chosen  angular  distance  away  from  the  center.  With  this  test  we  have 
shown  that  we  can  reliably  estimate  the  angular  field  in  which  the  paired  forms  are  recognized, 
and  can  use  the  position  and  width  of  that  field  referred  to  the  center  of  gaze  as  qualifying  the 
strategy  of  visual  perception  in  immediate  use. 

For  it  is  the  case  that  the  “spotlight  of  attention”  can  expand  or  constrict  and  change  its  center  of 
gaze  depending  on  the  cognitional  strategy. 

This  is  the  test  we  expect  to  modify  in  order  to  check,  in  a  non-interventive  way,  the  strategic 
competence  of  visual  perception  in,  say,  a  flight  controller  or  a  pilot.  The  stimuli  are  transient, 
involve  occasional  momentary  random  changes  in  motion  of  objects  on  a  screen  and  identifying 
the  direction  and  magnitude  of  such  changes.  We  believe  that  this  relatively  simple  test, 
conceptually  speaking,  will  do  more  to  reveal  the  running  visual  cognitive  competence  of  an  active 
observer,  than  would  any  physical  measures  on  the  eyes. 

EXPERIMENTAL  PROTOCOLS 

FIRST  PROTOCOL:  COMPARING  THE  FORM-RESOLVING  FIELDS  (FRF)  OF  JAGGED  AND 
SMOOTH  LETTERS 

Methods  and  Apparatus 

Three  slide  projectors,  equipped  with  flat  field  lenses  and  electrically  activated  shutters  (Vincent 
Associates)  back-projected  images  of  letters  on  a  diffusing  screen.  The  arrangement  served  as  a 
wide-screen  tachistoscope.  Each  of  the  projectors  gave  uniform  luminance  of  210  cd/m  +10% 
across  the  whole  screen.  The  first  projector  carried  a  slide  with  a  central  fixation  point.  The 
second  carried  the  stimulus  slide.  The  third  carried  a  blank  "eraser"  slide.  The  order  of 
presentation  of  the  slides  on  the  screen  and  the  duration  of  presentation  was  controlled 
electronically  by  a  timer.  It  adjusted  the  openings  and  closings  of  the  shutters  to  minimize 
transitional  changes  in  luminance  on  the  screen  between  presentations.  Stimulus  duration  was 
adjustable  over  the  range  of  1 .6  ms  to  1 50  ms.  The  stimulus  duration  for  each  subject  was 
individually  chosen,  as  will  be  explained  in  the  procedure  section.  The  screen  was  49  cm  wide 


and  32  cm  high  corresponding  to  390  of  visual  angle  horizontally  and  260  vertically  from  an 
observation  distance  of  69  cm. 

Stimuli 

Each  stimulus  slide  carried  two  letters,  one  in  the  center  at  the  location  of  the  fixation  point  and  the 
second  in  the  periphery.  The  stimuli  were  divided  into  5  groups.  In  each  group  the  letters  in  the 
periphery  were  at  a  fixed  angular  horizontal  distance  from  the  fixation  point  at  the  center.  The 
angles  used  were  from  2.50  to  12.50  in  2.50  steps.  Each  group  contained  40  slides.  In  half  of 
them  the  peripheral  letter  was  to  the  left  of  the  fixation  point  and  in  half  to  the  right  (i.e.  there  were 
10  slides  at  each  eccentricity).  They  were  presented  in  a  random  order.  This  procedure  helped 
observers  to  maintain  central  fixation  and  reduced  the  bias  of  expectation  of  appearance  (Geiger 
and  Lettvin,  1989,  Geiger  et  al.,  1992).  The  two  letters  on  each  stimulus  presentation  were 
different,  and  were  chosen  from  a  fixed  set  of  10  upper-case  Helvetica-medium  letters  (I,  S,  C,  O, 
V,  M,  N,  E,  T,  H). 

We  made  two  sets  of  slides  of  identical  letters  with  identical  eccentricities.  The  difference  between 
the  sets  was  that  in  one  the  letters  were  smooth,  as  they  appear  in  print  and  in  the  other  they  were 
jagged  as  they  might  appear  on  a  CRT  screen.  The  sets  were  otherwise  identical  in  their  font,  size 
and  stroke  width.  Each  letter  in  a  set  appeared  once  at  each  of  the  eccentric  visual  field  positions, 
and  twice  at  the  central  position.  The  letter  height  subtended  35  min  of  visual  arc,  and  letter 
contrast  was  90%. 

Subjects 

Seven  adult  subjects  from  the  university  population  were  measured  in  this  section  of  the  study. 
Their  ages  were  22  to  55  years  of  age.  All  were  ordinary  (proficient)  readers,  and  all  had  normal, 
or  corrected  to  normal  vision. 

Procedure 

Each  subject  sat  69  cm  away  from  the  screen  in  a  dimly  lit  room.  A  fixation  point  was  projected  on 
the  screen.  The  subject  was  asked  to  look  at  the  fixation  point.  Shortly  after  a  verbal  warning  the 
stimulus  slide  was  briefly  projected  (replacing  the  fixation  point  slide)  and  followed  by  a  blank 
eraser  slide  which  was  projected  for  2.5  s.  The  subject  was  asked  to  name  the  letters  in  the 
stimulus  slide  and  their  relative  position.  This  cycle  continued  until  all  160  stimuli  slides  had  been 
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presented.  The  effective  stimulus  duration,  the  time  between  the  offset  of  the  fixation  point  slide 
and  the  onset  of  the  eraser  slide,  was  determined  for  each  subject  separately  prior  to  the  test 
itself.  Various  stimulus  duration  were  tested  (with  samples  of  the  letter  pairs)  in  this  pre-test  until  a 
stimulus  duration  was  found  in  which  the  correct  letter  identification  reached  just  1 00%  at  one 
eccentricity  of  the  peripheral  letter  in  the  display  (this  was  at  2.50  for  all  the  subjects).  Once  the 
appropriate  stimulus  duration  was  found  it  was  kept  constant  during  the  test  and  for  the 
subsequent  FRF  measure.  After  all  stimulus  slides  were  presented  the  average  letter  recognition 
at  each  eccentricity  was  calculated  and  plotted  to  give  the  FRF  plot.  Separate  FRF  plots  were 
made  for  the  jagged  letters  and  for  the  smooth  letters,  and  were  then  compared. 

FRF  Study  Results 

The  form-resolving  field  (FRF)  is  the  plot  of  average  correct  recognition  as  a  function  of 
eccentricity  of  the  peripheral  letters  of  letter  pairs,  where  one  of  the  letter  pairs  was  at  the  center  of 
gaze  and  the  other  in  the  periphery  along  the  horizontal  line. 

Figure  1  depicts  the  average  FRF  obtained  by  presenting  smooth  letters  (solid  line)  and  that  of 
jagged  letters  (dashed  line).  As  is  evident  from  the  figure,  jagged  letters  are  recognized  better  in 
the  periphery  than  are  smooth  letters.  This  difference  is  statistically  significant  (p<0.01)  except  at 
the  far  left  (-12.5  deg),  where  the  difference  is  not  statistically  significant.  The  recognition  of  the 
letters  at  the  center  of  gaze  was  100%  correct  for  both  letter  types. 

The  average  stimulus  presentation  time  was  5.4  ms.  (range  of  4.5  to  6.5  ms.).  As  previously 
mentioned,  the  stimulus  presentation  time  was  determined  individually  prior  to  the  measurement, 
and  then  kept  constant  throughout  the  measurements. 

We  did  not  measure  the  jagged  letters  at  2.5  and  5  deg.  distances  of  the  peripheral  letters,  as  this 
would  have  made  the  program  of  measurement  too  long  to  discount  fatigue  effects  on  the 
measurements.  However  as  the  recognition  of  the  letter  at  the  center  of  gaze  was  equal  for  all 
letters  we  feel  that  that  was  a  reasonable  compromise. 

The  FRF  of  blurred  letters 

In  order  to  exclude  the  notion  that  additional  higher  spatial  frequencies  in  the  jagged  letter 
(aliasing)  are  the  reason  for  the  superior  recognition  in  the  periphery,  we  conducted  a  second 


measurement  with  all  letters  (smooth  and  jagged)  being  blurred  by  2  diopters.  We  achieved  the 
blurring  by  defocusing  the  slide  projector  that  projected  the  letters.  The  procedure  of  the 
measurement  was  otherwise  like  that  of  the  previous  measurement  with  the  same  stimulus 
presentation  duration. 

The  results  were  surprising.  The  recognition  of  the  blurred  letters  in  the  periphery  (i.e.  7.5  deg  and 
farther  away)  was  similar  to  that  of  the  distinct  letters,  for  both  letter  types  (as  seen  in  Figure  2  for 
smooth  letters).  That  is,  the  superiority  of  peripheral  recognition  of  the  jagged  letters  over  the 
smooth  ones  remains  also  when  the  letters  are  blurred,  although  the  higher  spatial  frequencies  are 
diminished.  In  addition,  smooth  letters,  either  distinct  or  blurred,  are  recognized  equally  well  in  the 
periphery,  and  this  holds  for  the  jagged  letters  as  well.  (However,  the  smooth  letters,  when 
presented  closer  to  the  center,  are  recognized  better  when  distinct.) 

Detecting  whether  letters  are  jagged  or  smooth 

The  question  remained  if  higher  cognitive  access,  i.e.  recognizing  the  letter  type  (smooth  or 
jagged),  could  have  influenced  recognition.  To  answer  this  question  we  presented  the  subjects 
with  all  the  letters,  as  before,  with  the  same  stimulus  presentation  time,  but  rearranged  their  order 
of  presentation.  The  subject  were  to  tell  only  if  the  letters  they  saw  were  jagged  or  smooth, 
without  identifying  each  letter.  The  default  answer  by  all  the  subjects  was  that  the  letters  were 
smooth.  Only  about  7%  of  the  jagged  letters  were  correctly  detected  as  such  in  the  periphery 
(from  about  50%  of  all  letters),  however,  22%  were  correctly  identified  at  the  center  of  gaze. 

Discussion  of  Form  Resolving  Field  Experiments 

In  order  to  demonstrate  how  sensitive  the  FRF  measure  is  to  different  visual  strategies  we 
demonstrated  the  difference  in  the  FRF  to  smooth  and  to  jagged  letters.  As  we  demonstrated, 
recognition  of  jagged  letters  is  significantly  better  than  smooth  letters  in  the  near  peripheral  visual 
field.  As  we  also  showed,  this  is  not  due  to  aliasing  or  other  cognitive  effects  due  to  recognition  of 
the  difference  in  letter  types,  for  that  is  the  outcome  of  the  blurred  letter  measurement  and  the 
experiment  on  detection.  We  suggest  however,  that  the  difference  in  recognizing  the  letter  type  is 
due  to  different  visual  strategies  employed  by  the  subjects  when  viewing  smooth  or  jagged  letters. 
A  crude  demonstration  in  perception  is  given  in  Figure  3.  When  looking  at  the  large  (dizzying) 
print  one  feels  differently  than  when  looking  at  the  smooth  smaller  print.  When  moving  ones  gaze 
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from  one  to  the  other,  the  individual  is  most  aware  of  the  shift.  This  difference  is  not  due  to  the 
size  of  the  letters,  but  rather  the  manner  in  which  they  appear. 

As  we  have  previously  suggested  (Geiger  et  al.  1992),  a  visual  strategy  is  determined  first  by  the 
region  of  distinct  vision  which  is  necessary  for  the  accomplishment  of  the  task.  This  region  of 
distinct  vision  is  in  the  foreground  whereas  the  remainder,  which  is  not  crucial  to  the  task,  is  in  the 
background,  or  ambient  visual  texture.  We  suggested  that  what  relegates  a  visual  figure  to  the 
background  is  an  active  masking  process  (Geiger  and  Lettvin,  1986,  Geiger  et  al.  1992),  which  is 
known  in  the  literature  as  lateral  masking.  Lateral  masking  is  the  adverse  effect  on  recognition  that 
neighboring  visual  elements  have  on  each  other.  A  demonstration  of  lateral  masking  is  presented 
in  figure  4.  For  additional  references  on  lateral  masking  see:  Bouma  1970,  Mackworth  1965, 
Townsend  et  al.  1971  and  Wolford  and  Chambers  1983).  As  can  also  be  seen  from  the  figure 
demonstrating  lateral  masking,  it  reduces  a  coherent  well  ordered  form  to  a  clumped  aggregation 
of  the  parts  (Geiger  et  al  1992).  Furthermore,  we  have  demonstrated  that  the  FRF  measure 
reflects  faithfully  the  measure  of  the  distribution  of  lateral  masking  over  the  visual  field  (Geiger  and 
Lettvin  1987,  Geiger  et  al  1992).  We  also  have  shown  that  the  distribution  of  lateral  masking 
changes  with  the  task  (Zegarra-Moran  and  Geiger,  1993). 

We  demonstrated  in  our  earlier  work  that  a  perceptual  strategy  is  learned  by  practice,  and  is  very 
specific  for  the  task.  That  is,  one  has  to  learn  a  new  strategy,  not  simply  modify  by  slight  change 
to  an  existing  one  (Geiger  and  Lettvin,  1987,  Geiger  et  al  1992). 

In  these  terms  we  suggest  that  while  viewing  jagged  letters  the  lateral  masking  nearer  to  the 
center  is  reduced  as  compared  with  viewing  smooth  letters  where  lateral  masking  is  more  active 
closer  to  the  center  of  gaze.  To  demonstrate  the  validity  of  this  argument  we  have  shown  a  string 
of  3  smooth  letters  in  the  near  periphery  (Figure  5)  to  38  persons  and  asked  them  to  recognize 
(away  from  a  fixation  point)  the  middle  letter  in  the  peripheral  string  and  then  did  the  same  with  the 
same  letters  jagged.  In  all  cases  the  persons  asked  found  it  easier  to  recognize  the  middle  letter 
in  the  string  when  the  letters  were  jagged. 

A  remarkable  case  in  point  involved  an  apparent  experimental  appearance  of  "fatigue"  which  was 
accompanied  with  radical  change  of  the  FRF  and  task  performance  is  a  "conditional  dyslexic" 
person  (Geiger  et  al.  1992).  This  person  was  able  to  read  normally  and  perform  similar  visual 
tasks  in  the  morning  when  "fresh".  However,  in  the  afternoon  when  he  was  "fatigued"  he  was 
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unable  to  read  (only  able  to  skim  the  text  fast  and  very  inaccurately)  but  was  a  very  able  and 
precise  artist  using  non-textual  visual  strategies.  His  FRFs  of  the  morning  and  afternoon  phases 
are  shown  in  Figure  6.  He  had  readers'  FRF  in  the  morning,  but  in  the  afternoon  he  possessed  a 
broad  FRF  with  masking  near  the  center  right  (which  is  characteristic  to  dyslexics)  (Geiger  and 
Lettvin,  1987, 1993,  Geiger  et  al  1992)  in  the  fatigued  phase. 

These  sets  of  measurements  demonstrate  the  task-specific  distinctions  between  visual  strategies, 
and  the  efficiency  of  the  FRF  as  a  measure  to  detect  such  differences. 

In  the  light  of  these  findings  and  our  previous  publications,  we  suggest  that  any  measure  for  the 
level  of  task  performance  must  be  specific  for  each  task.  We  furthermore  believe  that  the  FRF  or  a 
modified  form  of  it  is  the  appropriate  candidate  for  detecting  cognitive  overload  by  detecting  the 
loss  of  the  capacity  to  continue  to  perform  a  specific  task  at  the  skill  level  required. 

An  additional  aspect  of  our  experimental  results  that  demonstrate  a  wider  visual  strategy  for 
jagged  letters  concerns  the  use  of  CRT  screens.  CRT  screens  (unless  they  are  of  the  ultra¬ 
modern  flat  screen  variety)  do  not  display  letters  with  optically  smooth  edges.  Since  CRT 
displayed  characters  will  be  of  the  jagged  variety,  this  means  that  the  ability  to  localize  ones 
attention  on  a  CRT  screen  will  be  reduced  compared  with  similar  visual  icons  presented  optically 
with  smooth  edges.  That  is  expressed  commonly  when  proof  reading  a  text.  This  explains  why 
most  editors  prefer  reading  from  the  print  rather  than  from  the  CRT  screen. 

SECOND  PROTOCOL:  MEASUREMENT  OF  EYE-MOVEMENTS  WHILE  OBSERVING 
RANDOM  DOT  STEREOGRAMS  (RDS) 

Methods  and  Apparatus 

Binocular  eye-movements  of  the  subjects  were  recorded  with  an  ET3  eye  tracking  system 
(AMTech  GmbH)  which  is  a  binocular  eye-tracker.  The  technical  details  of  this  device  are 
presented  in  the  appendix  of  this  report.  The  eye-tracker  simultaneously  records  the  horizontal 
and  vertical  position  of  each  eye  separately.  These  channels  allow  the  plotting  of  the  individual 
and  conjugate  action  of  the  eyes  in  1  or  2  dimensions.  Thus  the  horizontal  vergence  response  can 
be  retrieved  by  plotting  the  running  differences  of  the  horizontal  positions  of  both  eyes  as  a 
function  of  time. 
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The  horizontal  position  of  each  eye  was  measured  100  times  each  second,  while  the  vertical 
position  50  times.  At  that  frequency  of  recording  the  eye-tracker  had  a  horizontal  spatial  resolution 
of  6  min  of  arc  and  a  vertical  spatial  resolution  of  18  min.  of  arc. 

An  additional  two  signals  were  incorporated  into  the  eye-tracker  system:  that  of  the  stimulus  (from 
begin  to  end  of  stimulus  presentation),  and  that  of  stimulus  recognition  (or  absence  of  recognition) 
by  the  subject.  A  subject  held  push-button  that  was  pressed  when  the  form  presented  was 
recognized  (or  partially  recognized)  indicated  stimulus  recognition. 

On  the  half  mirror  of  the  eye-tracker  through  which  the  subject  looked,  we  placed  a  red  filter  in 
front  of  one  eye  and  a  green  one  in  front  of  the  other,  in  such  a  way  that  each  eye  saw  the 
stimulus  by  one  color  only. 

Stimuli 

The  stimuli  consisted  of  different  forms  incorporated  in  random  dot  stereograms  (RDS).  These 
stereograms  were  consisted  of  two  images  composed  of  random  dots,  one  presented  in  red  and 
the  other  in  green.  The  random  dots  in  the  two  halves  are  identical  except  for  the  form  to  be  seen, 
say  a  square,  where  there  the  dots  are  shifted  slightly  (disparate)  in  position.  This  disparity  is 
processed  by  the  person's  nervous  system  to  be  seen  as  a  square  surface  on  a  different  plane 
from  the  rest  of  the  surface  of  the  dot  array.  The  form  is  perceived  by  virtue  of  its  apparent  depth 
with  respect  to  the  background. 

Two  slide  projectors  back  projected  the  two  dot  arrays  of  the  RDS  onto  the  same  location  on  a 
semi-translucent  screen.  In  front  of  one  slide  projector  there  was  a  red  filter  and  a  green  filter  was 
placed  in  front  of  the  other.  This  arrangement  gave  a  crossed  disparity  when  a  red  filter  was  in 
front  of  the  right  eye  and  a  green  filter  in  front  of  the  left  eye  of  the  subject.  The  RDS  was  viewed 
from  the  side  of  the  screen  opposite  to  the  projectors.  The  crossed  disparity  results  in  the  percept 
of  the  form  being  towards  the  subject.  When  the  color  filters  in  front  of  the  eyes  were  reversed, 
the  uncrossed  disparity  gave  the  percept  of  the  form  being  further  away  from  the  subject. 


The  size  of  the  screen,  the  distance  of  the  slide  projectors  and  the  distance  of  the  subject  from  the 
screen  were  so  arranged  to  give  the  stimulus  a  field  of  22.1  deg.  of  visual  arc  horizontal  and 
vertical.  With  that  arrangement  a  dot  size  (of  the  RDS)  was  3.2  min  of  visual  arc. 

There  were  a  few  different  forms  embedded  in  different  stimuli.  They  were: 

>  A  flat  field,  where  the  whole  field  size  was  at  one  or  another  disparity  to  give  the  percept  of  a 
large  rectangular  planar  surface. 

>  A  small  square  (5.5  deg.  x  5.5  deg.)  in  the  center. 

>  A  medium  size  square  (8  deg  x  8  deg.)  in  the  center. 

>  A  concentric  double  square  in  the  center,  like  a  two-layered  concentric  pyramid  with  a  5.5  x  5.5 
deg.  square  on  top  of  an  8  x  8  deg.  square. 

>  Three  stairs  from  right  to  left  along  the  height  of  the  field 

>  A  concentric  spiral. 

Three  dots  densities  were  used  in  different  stimuli,  namely  5000, 10000  and  20000  dots  over  the 
whole  field.  The  contrast  of  the  dots  was  approximately  50%. 


Subjects 

Eight  adults  from  the  university  population  were  tested.  They  were  22  to  36  years  of  age.  All  the 
subjects  except  one  had  normal  stereo  vision.  The  one  different  was  "stereo  blind",  i.e.,  he  could 
not  perceive  the  forms  presented,  even  over  very  long  viewing  periods.  Otherwise,  all  subjects 
had  normal  vision. 

Procedure 

The  subjects  were  first  tested  for  their  stereo  vision  by  viewing  the  RDS  stimuli  through  the  red 
and  green  filters  but  not  yet  harnessed  to  the  eye-tracker. 

After  their  stereo  vision  was  tested  they  were  asked  to  look  at  the  screen  through  the  eye-tracker, 
while  the  apparatus  was  adjusted. 


Once  adjusted,  a  calibration  record  was  taken.  The  subject  was  asked  to  gaze  at  9  successive 
locations  on  the  screen  while  the  eye-tracker  recorded  the  movements  of  the  eyes.  These  points 
consisted  of  a  fixation  point  along  with  eight  symmetrically  arranged  dots  covering  the  rest  of  the 
field.  A  third  slide  projector  projected  these  dots.  This  record  provided  a  baseline  for  the 
horizontal  and  vertical  ocular  response  to  known  angular  distances. 

A  second  calibration  record  was  conducted  for  vergence  response.  The  subject's  eye-movements 
were  recorded  while  moving  the  gaze  from  the  center  fixation  point  on  the  screen  to  central  closer 
points,  held  by  an  optical  bench  closer  to  the  subject  by  25  cm  and  50  cm.  These  points 
corresponded  to  46  min.  of  arc  and  2  deg  of  arc  changes  from  the  zero  position,  which  is  that 
when  the  eyes  gaze  on  the  fixation  point  on  the  screen. 

After  the  calibration  records  were  made  for  each  subject,  the  recording  of  eye-movements  while 
viewing  RDS  began. 

At  first  a  fixation  point  was  projected  on  the  screen  (projected  by  the  third  slide  projector).  The 
subject  was  asked  to  fix  the  gaze  at  that  point.  Then  the  recording  of  the  eye-movements  began. 
After  2  seconds  the  fixation  point  disappeared  and  the  two  half  images  appeared.  The  exchange 
took  less  than  3  ms.  and  was  controlled  by  fast  electronic  shutters.  The  subject  was  instructed 
before  the  recording  to  freely  move  the  eyes  when  presented  a  stimulus.  The  stimulus  remained 
on  the  screen  for  10  s.  after  which  the  fixation  point  re-appeared.  The  onset  of  the  stimulus  and  its 
termination  was  recorded  also  by  the  eye-tracker.  The  subject  was  given  a  hand-held  push-button 
at  the  beginning  of  the  session,  and  instructed  to  press  it  as  soon  as  he  or  she  recognized  (or 
partially  recognized)  a  form.  This  signal  was  also  recorded  by  the  eye-tracker.  When  the  fixation 
point  re-appeared  at  the  end  of  stimulus  presentation  the  subject  was  instructed  (before  hand)  to 
gaze  at  that  point.  This  was  in  order  to  assure  that  there  was  no  drift  of  the  device  throughout  the 
recording.  When  the  record  was  concluded,  the  subject  was  asked  to  report  the  percept,  the  form 
seen  and  whether  it  appeared  towards  or  away  from  the  subject  with  respect  of  the  plane  of  the 
screen.  Following  this  debriefing,  a  new  stimulus  was  prepared  and  a  new  record  made.  The 
records  were  stored  in  the  eye-tracker's  computer  for  later  processing. 

Data  Processing 

Each  record  was  analyzed  for  the  spatial  and  temporal  relations.  They  included: 


>  Position  detection  by  the  eyes  (the  instant  the  saccade  comes  within  1  deg.  of  the  boundaries 
of  the  form).  This  estimate  was  made  possible,  as  the  location  of  the  form  on  the  screen  was 
known.  Due  to  the  calibration  record  the  relative  eye-position  given  by  the  eye-tracker  could 
be  reliably  related  to  the  form's  location  and  the  location  of  its  contours. 

>  Vergence  response  (the  instant  the  horizontal  vergence  response  changes  to  reach  a  new 
steady  plane).  The  horizontal  vergence  was  measured  by  the  difference  of  the  angular 
horizontal  positions  of  the  two  eyes.  The  vergence  at  the  fixation  point  determines  the 
reference  (zero  depth).  Any  change  gives  either  a  negative  number  taken  as  crossed 
response  (towards  the  subject)  or  a  positive  number  taken  as  an  uncrossed  response  (away 
from  the  subject). 

>  The  instant  of  recognition  of  the  form.  (If  the  form  was  not  recognized  throughout  the  1 0  s.  of 
viewing  time  of  the  stimulus,  that  instant  was  considered  to  be  10s). 

In  this  way  we  could  determine  the  temporal  differences  between  the  detection  by  the  eyes  of  the 
attributes  of  the  forms  (position  and  vergence)  separately  from  the  recognition  of  the  form. 

As  will  be  presented  later,  statistical  averaging  of  the  time  required  for  detection  or  recognition  was 
conducted. 

In  addition,  correlations  were  made  for: 

s  The  form  presented  with  the  form  perceived. 

✓  The  depth  perceived  with  the  actual  disparity. 

•/  The  recorded  vergence  with  the  actual  disparity. 
s  The  vergence  response  with  the  perception  of  depth. 

Results  of  the  Eye-Tracking  Experiments 
Single  Eye-Movement  Records 

The  results  from  individual  eye-movement  records  were  broad  in  their  response.  Records  showing 
saccadic  eye-movements  ranged  from  one  in  which  the  eyes  moved  to  the  position  of  the  form's 
contour  accompanied  by  an  appropriate  vergence  response  notwithstanding  the  fact  that  the  form 
was  not  (even  partially)  recognized  by  the  subject.  This  response  is  shown  in  Figure  7.  On  the 
other  extreme,  another  record,  presented  in  Figure  8,  shows  the  eye  movements  of  the  same 
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subject  (same  session)  viewing  the  same  RDS  square  form  as  before.  In  this  case,  however,  the 
subject  recognized  the  form  before  any  directed  saccadic  or  vergence  eye-movements  were 
made.  After  recognizing  the  form,  the  subject’s  eyes  moved  to  the  form's  contours  and  made  the 
appropriate  vergence  response. 

To  demonstrate  that  the  saccadic  movements  of  the  eyes  were  not  completely  random,  but  had 
been  related  to  the  form's  attributes  we  show  two  more  records  in  Figures  9  and  10.  In  these 
records  the  stimuli  were  presented  to  the  same  subject  (and  in  the  same  session)  but  they  were 
only  flat  surfaces  subtending  the  whole  RDS  field  which  were  either  closer  (Figure  9)  or  further 
(Figure  10)  from  the  subject  with  relation  to  the  plane  of  the  fixation  point.  It  is  evident  that  in  both 
of  these  records  the  saccadic  eye-movements  were  very  different  from  the  previous  records,  since 
there  were  no  form  contours  present.  In  these  records,  as  in  the  previous  ones,  the  vergence 
response  was  appropriate. 

The  eye-movement  records  have  consistently  shown  that  in  the  presence  of  contours  in  the  stimuli 
there  were  directed  saccades  towards  them  (or  within  a  short  angular  distance  from  them); 
similarly,  with  the  vergence  response.  That  allowed  us  to  assume  that  when  the  eyes  made  these 
movements  they  "detected"  the  form's  attributes  in  these  stimuli.  Once  we  made  this  assumption, 
we  could  consider  the  first  directed  saccade  to  the  contour's  position  as  the  instant  of  detection  of 
the  contour's  position  by  the  eyes.  Similarly  with  the  vergence  response.  That  is,  the  first  instant 
at  which  the  steady  vergence  response  is  in  the  direction  given  by  the  stimulus'  disparity,  is  a  sign 
of  correct  detection  of  that  disparity. 

A  short  technical  remark  is  appropriate  here.  In  general,  when  that  which  is  displayed  to  a  subject 
is  abruptly  changed  (as  in  our  experiments,  the  change  from  the  fixation  point  to  the  RDS  stimuli), 
this  change  resulted  in  a  -0.5  deg.  automatic  (reflexive)  vergence  response,  i.e.,  towards  the 
subject.  Once  that  adjustment  reflex  response  was  made  the  actual  zero  vergence  is  then  at  -0.5 
deg.  Hence,  we  took  in  to  account  this  factor  in  making  the  vergence  response  calculations. 

Most  of  the  eye-movement  records  we  measured  contained  similarities,  but  varied  considerably. 
We  therefore  characterized  the  ensemble  of  the  records  with  a  statistical  treatment  of  the  data. 

Statistical  Analysis  of  Some  Eye-Movement  Parameters 
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As  previously  mentioned,  we  calculated  4  correlation  factors  and  3  different  temporal  measures 
from  each  record.  The  averages  of  these  measurements  are  presented  in  Table  1. 

The  same  letter  that  appears  on  top  of  the  respective  column  in  Table  1  indicates  the  correlations 
and  temporal  factors  listed  here. 

These  correlations  are: 

B.  The  form  presented  with  the  form  perceived. 

C.  The  depth  perceived  with  the  actual  disparity. 

D.  The  recorded  vergence  with  the  actual  disparity. 

E.  The  vergence  response  with  the  perception  of  depth. 

When  the  parameters  were  in  agreement,  the  response  was  assigned  the  numeral  1;  when  the 
response  signified  misperception  or  that  the  parameters  were  in  disagreement,  the  response  was 
assigned  the  numeral  0. 

The  temporal  factors  are: 

G.  The  time  taken  from  the  onset  of  the  stimulus  to  the  recognition  of  the  form  by  the 
subject. 

H.  The  time  taken  from  the  onset  of  the  stimulus  until  a  directed  saccade  was  made  to 
within  1  deg.  of  the  form's  contour. 

I.  The  time  taken  from  the  onset  of  the  stimulus  to  a  registered  steady  vergence 
response. 


Discussion 

From  the  data  presented,  our  main  argument  is  that  stimulus  recognition  and  eye-movement 
parameters  are  poorly  correlated,  hence  eye-movements  are  a  poor  measure  of  cognition. 

The  RDS  records  shown  in  Table  1  demonstrate  that  only  in  66.46%  of  the  records  did  the 
subjects  recognize  the  forms  embedded  in  the  RDS  stimuli.  Furthermore,  the  subjects  were  able 
to  correctly  perceive  the  depth  information  embedded  in  the  stimuli  only  44.94%  of  the  time.  At  the 
same  time,  the  eyes  were  able  to  perform  82.91%  correct  vergence  response  to  the  given 
disparities  in  the  stimuli.  In  addition,  only  in  45.57%  of  the  cases  were  the  vergence  response  and 
the  subjects'  depth  perception  in  agreement.  These  numbers  alone  indicate  poor  correlations 
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between  recognition  and  eye-movements  where  eye-movements  are,  by  far,  more  reliable  in 
locating  the  stimulus. 

This  finding  is  further  emphasized  when  we  only  consider  the  records  wherein  the  subjects  did  not 
recognize  the  forms  at  all  (about  a  third  of  the  cases  we  recorded).  In  these  cases  83.33%  of  the 
records  have  shown  correct  vergence  response  by  the  eyes  while  only  in  2.08%  of  these  stimuli 
did  the  subjects  perceive  the  depth  correctly. 

More  telling  are  the  temporal  factors.  Averaging  over  all  eye-movement  records,  it  took  the 
subjects,  on  average  5.05  seconds  to  recognize  the  forms  in  the  RDS,  whereas  it  took  the  eyes 
only  1 .48  seconds  to  detect  the  contours  of  the  forms  and  1 .51  seconds  to  make  a  correct 
vergence  response.  The  difference  in  duration  between  the  signs  of  detection  and  the  event  of 
recognition  cannot  be  explained  by  a  motor  response  delay  caused  by  pushing  the  push-button. 

To  elaborate  further,  when  averaging  all  cases  where  the  time  for  recognition  was  3  seconds  and 
longer,  we  found  that  it  took  on  average  7.98  seconds  for  recognition  but  only  1.69  seconds  to 
detect  the  contours  and  1 .82  seconds  to  make  the  correct  vergence  response.  In  the  cases  where 
the  forms  were  not  recognized  at  all  by  the  subjects,  the  average  time  it  took  the  eyes  to  detect  the 
contours  was  2.36  s.  and  2.22  s.  for  correct  vergence.  The  time  for  the  eyes  to  detect  the  attribute 
of  the  presented  forms  is  very  small  in  comparison  with  the  time  that  was  required  for  form 
recognition. 

This  data  strongly  supports  the  notion  that  eye-movements  do  not  correlate  with  cognition. 

In  addition,  as  we  mentioned  earlier,  one  of  the  subjects  was  stereo  blind,  and  could  not 
recognized  the  forms  at  all.  However,  his  eyes  tracked  the  attributes  of  the  forms  faithfully. 


GENERAL  DISCUSSION  AND  CONCLUSION 

In  summary,  our  two  experimental  protocols  bear  directly  on  relating  external  measures  to  visual 
cognition,  in  particular  with  a  view  to  detecting  signs  of  cognitive  overload,  fatigue  or  insufficiency. 

>  The  study  on  eye-movements  versus  recognition  in  the  case  of  random  dot  stereograms 
revealed  the  fact  that  visual  cognition  and  eye-motion  are  not  so  coupled  that  one  can  be 


related  to  the  other  even  crudely.  Appropriate  eye  motion,  as  if  seeing  an  object,  occurred 
often  without  any  subjective  experience  of  seeing  it.  Conversely,  recognition  of  an  object 
occurred  well  before  the  eyes  moved  to  home  on  the  contours.  Since  the  goal  is  to  assure  that 
cognition  serves  delicate  and  appropriate  task  performance,  it  is  evident  from  this  work  that 
eye  motion  does  not  suffice  as  a  crude  measure  of  visual  cognition,  much  less  a  reliable  one. 

>  More  appropriate  may  be  direct  interactive  measures  that  test  an  individual’s  particular  visual 
strategy  in  the  use  at  the  moment.  Since  there  are  several  such  strategies  that  can  be 
characterize  in  terms  of  the  angular  width  and  shape  of  the  form-resolving  field  and  the 
centering  of  that  field  with  respect  to  the  center  of  gaze,  any  measure  which  reveals  an 
inappropriate  strategy  or  the  inflexibility  of  shifting  to  a  different  strategy,  is  more  likely  to  be  an 
index  of  cognitive  overload  or  fatigue  than  any  purely  external  measure.  The  FRF  is  the  pre- 
cognitive  setting  of  the  distribution  of  lateral  masking  for  the  task  at  hand. 

The  problem  of  assessing  blunted  cognition  involves  addressing  the  cognition  itself  as  inferred 
from  task-performance  rather  from  the  use  of  any  physical  measure  on  the  mechanism  alone.  As 
in  neurology  and  psychology,  external  signs  are  unreliable  until  function  is  tested,  in  a  direct  way. 
Work  reported  here  opens  a  new  path  to  interactive  tests  that  do  not  significantly  interfere  with  the 
task  being  performed.  These  tests  have  already  shown  their  value  in  diagnosis  and  treatment  of 
dyslexia,  a  disorder  that  cannot  be  measured  by  external  signs  alone  nor  by  tests  that  do  not 
challenge  the  cognitive  process  directly. 

It  is  the  development  of  such  carefully  targeted  methods  that  will  occupy  the  work  to  be  done  here, 
now  that  the  foundations  have  been  laid. 

POTENTIAL  COGNITIVE  AND  METHODOLOGICAL  TECHNIQUES  FOR  MEASURING 
COGNITIVE  OVERLOAD 

RESEARCH  OBJECTIVES 

♦  The  intended  scope  of  this  research  program  was  four-fold: 

♦  Determine  candidate  techniques  for  measuring  and  analyzing  ocular  behavior  as  an  index  of 
cognitive  workload  under  fatigue. 

♦  Experimentally  determine  a  relationship,  if  any,  between  ocular  behavior  and  fatigue-induced 
performance  decreases  in  tasks  of  human  visual  target  detection  and  target  tracking. 
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♦  Conceptually  explore  other  physiological  parameters  related  to  fatigue  or  stress  that  correlate 
with  ocular  behavior. 

♦  Provide  a  preliminary  design  specification  for  a  cognitive  workload  measurement  apparatus. 

FULFILLING  THE  OBJECTIVES 

Determining  the  Relationship  Between  Ocular  Behavior  and  Diminished  Task  Performance 

Our  experimental  protocols  have  made  clear  that  eye  movements,  per  se,  cannot  reflect  cognition. 

Hence,  techniques  for  measuring  and  analyzing  ocular  behavior  as  an  index  of  cognitive  workload 
under  fatigue  must  be  task  specific  and  pertain  to  the  individual  performing  said  task(s).  The  data 
processing  technology  for  determining  individual  performance  characteristics  is  state-of-the-art 
with  today’s  32  bit  embedded  microprocessor  technology.  Of  interest  is  the  nature  of  the  sensors 
used  to  determine  eye  movements  in  a  manner  that  is  comfortable  and  does  not  interfere  with  the 
user’s  ability  to  perform  the  task. 

Our  findings  can  be  summarized  as  follows: 

♦  The  continuous  recording  of  eye  movements  is  applicable  only  for  tasks  that  depend  upon  the 
monitoring  of  tracking  slow  moving  objects. 

♦  Eye  position  (i.e.,  the  determination  of  where  the  eyes  are  pointed)  can  be  detected  on  a  non- 
continuous  basis  for  applications  in  which  an  artificial  visual  stimulus  is  imposed  upon  the 
visual  task  and  eye  position  is  detected  in  addition  to  the  response  or  lack  of  response  of  the 
individual  performing  his  task.  In  such,  the  eye-tracking  sensor  (e.g.,  an  EOG)  is  turned  on  at 
the  time  the  artificial  stimulus  is  generated. 

♦  Each  visual  strategy  is  specific  to  the  task  performed  and  the  subject  performing  the  task  is 
well  practiced. 

♦  Norms  must  be  established  on  an  individual  basis  such  that  individual  deviations  that  reflect 
fatigue  or  cognitive  overload  can  be  detected. 
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Physiological  Measurement  Methods 


There  are  a  number  of  established  methods  of  measuring  eye  movements  and  eye  position.  They 
range  from  the  elegantly  simple  electro-oculogram  (EOG)  to  the  complex  infrared  or  video  camera 
pupilometry. 

EOGs  consist  of  four  miniature  bipolar  electrodes  placed  above,  below  and  to  each  side  of  each 
eye.  They  detect  the  electrical  potential  generated  by  the  dipole  of  the  eye  as  it  rotates  in  the 
socket.  In  addition  to  detecting  the  position  and  motion  of  each  eye  independently,  stereoscopic 
binocular  fixation  can  be  determined  by  manually  recorded  techniques  or  by  computer  analysis. 
EOGs  function  irrespective  of  eyelid  state,  and  eyelid  status  (open,  shut  or  blinking)  can  be 
determined  utilizing  the  same  electrode  used  to  measure  EOG.  The  EOG  is  very  useful,  therefore, 
in  determining  eye  motion  during  sleep  and  in  establishing  rapid  eye  motion  (REM)  sleep  states. 

EOG  designs  are  subject  to  amplifier  drift,  and  AC  coupling  techniques  used  to  establish  a 
consistent  baseline  undermine  attempts  to  observe  slow  processes  of  eye  drift. 

Pupilometers  are  very  exact  in  their  measurement  of  eye  position,  providing  the  eye  remains  open. 
Pupilometers  are  very  effective  in  counting  eye  blinks,  and  determine  pupil  size  in  addition  to  eye 
position.  Pupil  size  and  blinking  rates  might  be  useful  in  determining  sleepiness  and  fatigue,  and, 
thereby  measure  the  decrease  in  visual-based  performance. 

Pupilometers  are  complex  devices  and  are  difficult  to  use  in  normal  working  applications,  e.g., 
piloting  and  aircraft  or  monitoring  a  radar  screen.  They  are  usually  bulky,  expensive  and  require 
frequent  adjustment.  Recent  advances  in  microprocessor  technology,  laser  diode  technology  and 
ASIC  (application  specific  integrated  circuits)  fabrication  technologies  might  make  possible  the 
development  of  wearable  pupilometers,  however,  the  current  demand  for  such  technology  does 
not  justify  the  investment  required  to  make  such  devices  available  for  experimentation. 

Time-domain  measurements  of  respiration  and  cardiac  rates  have  come  into  vogue  in  recent  years 
and  have  been  used  to  measure  the  ability  of  an  individual  to  perform  tasks  under  physiologically 
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stressful  situations  (Krasner,  et  al.,  1979),  as  well  as  to  indicate  cardiovascular  compensations  to 
pathophysiological  changes  in  physiological  regulation  (e.g.,  early  warning  of  diabetic  or  vascular 
shock).  In  these  situations  cardiac  rate  is  measured  not  only  in  the  traditional  beats-per-minute, 
but  by  the  rate-of-change  (derivative)  of  the  time  period  between  heartbeats. 

It  is  known,  for  example,  in  diabetic  hypoglycemia  that  although  heart  rate  seldom  exceeds  85  to 
90  beats  per  minute,  the  normal  sinus  arrhythmia  associated  with  respiration  induced  vagal 
stimulation  gives  way  to  chaotic  changes  in  beat-to-beat  intervals.  It  may  be  possible  to  determine 
if  fatigue  alters  sinus  beat-to-beat  control  of  cardiac  rate  in  some  measurable  fashion. 

Preliminary  Design  Specification  for  a  Cognitive  Workload  Measurement  Apparatus 

Following  the  results  of  our  investigation,  we  provide  an  example  of  how  transient  measurement  of 
ocular  behavior  might  be  used  as  an  index  of  cognitive  workload  under  normal  and  fatigued 
conditions. 

The  following  represents  a  simplified  prototype  design  fortesting  cognitive  adequacy. 

In  many  of  the  tasks,  confronted  by  air  controllers  for  example,  a  detailed  and  accurate  account 
has  to  be  made  of  small  visual  icons.  As  an  example  of  testing  the  cognitive  adequacy  of  a  person 
at  that  task  we  suggest  the  following  prototype  procedure. 

The  task:  An  icon  is  presented  on  a  screen.  Its  quality  is  abruptly  changed  by  changing  the  color, 
contrast  or  sharpness  of  its  boundary  (not  by  moving  it).  The  subject's  task  is  to  press  a  push¬ 
button  as  soon  as  a  change  in  the  icon  occurs.  These  changes  are  made  intermittently.  While  the 
subject  attends  the  icon  and  presses  the  push-button  whenever  a  change  occurs,  a  letter  is  briefly 
flashed  (say  for  20  ms.)  at  about  10  deg.  of  visual  angle  to  either  the  right  or  left  side  of  the  icon, 
the  sideness  randomly  chosen.  If  the  subject  recognizes  that  letter,  this  indicates  that  the  attention 
is  wide  and  therefore  not  adequate  in  attending  only  to  the  icon.  But  if  the  letter  was  not  seen  by 
the  subject,  or  it  was  detected  by  the  subject  as  "something  that  was  there"  yet  unclear  what  it 
was,  that  will  imply  that  the  subject's  attention  is  narrow  and  pointed  at  the  icon.  The  cognitive 
adequacy  in  this  case  depends  on  the  ability  of  the  subject  to  attend  solely  to  the  icon.  If  desired, 
task  performance  could  be  gauged  to  verify  this  notion  by  measuring  the  subject's  reaction  times 
to  the  change  in  the  icon.  The  reaction  time  will  be  longer  when  the  attention  is  wide  and  shorter 
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(signifying  a  better  task  performance)  when  the  attention  is  only  on  the  icon.  This  is  one  example 
of  assessing  attention  and  hence  cognitive  adequacy.  The  method  can  be  incorporated  in  a  multi¬ 
task  situation. 

This  procedure  could  be  done  in  a  fraction  of  a  second,  and  hence  interfere  little  with  the 
performance  of  the  task  itself.  As  many  tasks  involve  and  depend  on  "pointed  attention"  this 
prototype  could  serve  as  the  test  of  cognitive  adequacy  while  performing  those  tasks.  However, 
as  tasks  differ  on  from  another  a  careful  and  detailed  design  will  have  to  be  made  for  separate 
kinds  of  tasks. 

There  also  are  tasks  where  visual  wide  field  has  to  be  attended.  In  a  similar  way  as  above,  letters 
(or  other  icons)  could  be  presented  in  the  center  field.  Their  recognition  would  indicate  narrow 
attention,  or  cognitive  inadequacy  for  the  task,  as  a  wide  attention  was  needed. 

These  prototypes  are  examples  derived  from  the  experimental  data  we  present  here  in  the  report. 
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Table  1,  RDS  general  results 
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2  3  I  Data  for  the  different  forms 
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Recognition  of  jagged  and  smooth  letters. 


Eccentricity  deg. 


Figure  1 .  The  average  FRF  measured  with  smooth  (solid  line)  and  jagged 
(dashed  line)  letters. 


Recognition  of  blurred  and  distinct  (smooth)  letters 


Eccentricity  deg. 


Figure  2.  The  average  FRF  measured  with  smooth  letters.  When  distinct  (solid 
line)  and  blurred  (dashed  line). 


A  demonstration  of  lateral  masking 
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Figure  4.  A  demonstration  of  lateral  masking.  Fix  your  gaze  on  the  upper  x. 
Without  shifting  your  gaze,  the  N  on  the  left  will  appear  clear  and  distinct 
whereas  the  N  on  the  right  will  not  be  legible  though  segment  lines  will  be  clear. 
Similarly  fix  your  gaze  on  the  lower  x.  You  will  see  small  circles  on  both  sides. 
Notice  that  on  the  left  side  there  is  a  spatial  arrangement  of  the  small  circles  (a 
ring  of  circles?)  while  on  the  right  the  arrangement  of  small  circle  is  lost. 
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Figure  5.  Fix  your  gaze  on  one  of  the  x’s.  Try  to  recognize  the  s  on  the  right.  All 
subjects  asked  recognize  the  jagged  s  easier. 


The  FRF  of  a  "conditional  dyslexic"  in  his  two  phases. 
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Figure  6.  The  FRF  measured  with  a  conditional  dyslexic.  The  dashed  line 
showe  his  FRF  when  he  was  in  the  alert  phase  and  the  solid  line  shows  the  FRF 
a  few  hours  later,  when  he  was  “fatigued”. 
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Figure  7.  A  single  eye-movement  record  of  a  subject  viewing  a  RDS  stimulus  with  a 
square  form  (indicated  by  a  dashed  line,  although  the  line  does  not  appear  in  the 
stimulus).  The  top  part  is  a  2-dimensional  plot  of  the  record.  The  lower  is  the 
horizontal  eye-position  record.  The  top  trace  is  of  the  right  eye,  the  lower  of  the  left. 
The  middle  trace  is  of  the  difference  between  the  position  of  the  eyes.  It  measures  the 
vergence  response. 
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Figures  8.  A  single  eye-movement  record  of  a  subject  viewing  a  RDS  stimulus  with  a 
square  form  (indicated  by  a  dashed  line,  although  the  line  does  not  appear  in  the 
stimulus).  The  top  part  is  a  2-dimensional  plot  of  the  record.  The  lower  is  the 
horizontal  eye-position  record.  The  top  trace  is  of  the  right  eye,  the  lower  of  the  left. 
The  middle  trace  is  of  the  difference  between  the  position  of  the  eyes.  It  measures  the 
vergence  response. 


Figure  9.  A  single  eye-movement  record  of  a  subject  viewing  a  RDS  stimulus  with  a 
flat  plane  (over  the  whole  stimulus)  pointing  towards  the  subject.  The  top  part  is  a  2- 
dimensional  plot  of  the  record.  The  lower  is  the  horizontal  eye-position  record.  The  top 
trace  is  of  the  right  eye,  the  lower  of  the  left.  The  middle  trace  is  of  the  difference 
between  the  position  of  the  eyes.  It  measures  the  vergence  response. 


Figure  10.  A  single  eye-movement  record  of  a  subject  viewing  a  RDS  stimulus  with  a 
flat  plane  (over  the  whole  stimulus)  pointing  away  from  the  subject.  The  top  part  is  a  2- 
dimensional  plot  of  the  record.  The  lower  is  the  horizontal  eye-position  record.  The  top 
trace  is  of  the  right  eye,  the  lower  of  the  left.  The  middle  trace  is  of  the  difference 
between  the  position  of  the  eyes.  It  measures  the  vergence  response. 
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_ AMTech  GmbH 

ET3  EYE  TRACKING  SYSTEM 


Calibration  Matrix 


the  AMTech  ET3  is  a  fast,  high  resolution  eye  tracking  unit  that 
determines  horizontal  and  vertical  eye  positions  by  using  a 
high  resolution  linear  CCD  camera.  This  technology  stands  for 
up  to  400  samples  per  second,  high  resolution,  excellent 
linearity  and  no  crosstalk  between  horizontal  and  vertical 
channels. 

The  ET3  also  connects  into  a  wide  range  of  equipment  for 
controlling  and  display  of  various  stimuli;  laser  projector  for 
slow  pursuit,  LED  displays  for  any  kind  of  saccade  paradigms 
and  video  displays. 

The  ET3  software  covers  all  needs  from  defining  stimulus 
paradigms  to  saccades  analysis,  pursuit  analysis  and  a 
sophisticated  fixations  analysis. 


Detail  from  FIXATION 
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ET3  SYSTEM  COMPONENTS 


horizontal, 
vertical, 
monocular  and 

binocular  systems  are  available 

high  resolution,  excellent  linearity, 
can  be  used  as  pupilometer 


Eye  Tracking  Unit: 

fast, 


l_aser  Taraet: 

-  '  1  or  2  dimensional  precision  laser  scanner 

including  control  card  and  software 


LED  Targets: 


16  programmable  digital  signals  for  LEDs, 
buzzers,  shutter  etc. 


Analog  output: 


output  of  data  for  on  line  display  manipulation  or 
chart  recorder 


Parallel  port: 


on  line  data  transmission  to  display  PC  or  other 
equipment 


ET3  software  provides : 

Stimulus  generator: 

uses  text  Tiles  to  make  stimuli  for  LEDs,  buzzer  and  Laser 
scanner 

•  Display  program: 

(in  preparation)  uses  list  of  PCX  files  and  push  button 
information  to  display  scenes  or  text  on  display  PC. 

Data  acquisition: 

collects  eye  movement  data  and  push  button  data 
controls  stimulus, 

can  transfer  eye  position  to  display  computer, 

„  analysis: 

Analysis  of  saccades  and  pursuit  ->  EYEMAP 
Analysis  of  fixation  ->  FIXATION 
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ET3  EYE  TRACKING  UNIT 


The  ET3  Eye  Tracker  is  available  from  monocular,  horizontal  only  to  binocular ,  horizontal 
and  vertical  recording. 

All  systems  are  capable  of  pupilometrle. 


SPECIFICATIONS 


range: 


linear  range: 


resolution: 


frequency: 

horizontal 

from  50  to 

+  vertical 

from  50  to 

horizontal 

+/-  40  0 

vertical 

+/-  8° 

ge: 

horizontal 

+/-  15  0 

vertical 

+/-  8° 

i: 

horizontal 

2  arcmin 

vertical 

10  arcmin 

Hz 

Hz 


trigonometrical  corrections  can  expand  the  linear  range  to  the  full  measuring  range. 


PRINCIPLES  OF  OPERATION 

The  eye  is  illuminated  by  infrared  light  (880nm).  It  is  them  imaged  on  to  a  linear  array  of 
1728  photo  diodes  (CCD-line  camera)  by  means  of  a  dichroic  mirror,  transparent  for  visible 
light  yet  reflective  for  infrared  light,  and  a  lens. 

The  CCD  array  of  photo  diodes  is  only  13  microns  high  and  therefore  puts  a  very  thin  line 
trough  the  image  of  the  eye  i.e.  the  pupil.  The  CCD  produces  a  video  signal  that  contains 
the  exact  position  of  the  pupil  margins.  This  position  is  transferred  to  a  PC  for  storage  and 
further  calculations.  As  the  position  of  both  pupil  margins  is  know,  it  also  possible  to 
calculate  the  diameter  of  the  pupil  precisely. 

The  vertical  component  of  the  eye  position  is  determined  by  placing  two  measuring  lines 
across  the  image  of  the  eye.  For  this  purpose  the  image  is  shifted  up  and  down  by  a 
galvanometer  mirror  altematinglv.  The  vertical  position  can  thus  be  calculated  from  two 
consecutive  data  sets,  representing  4  positions  on  the  circle  of  the  pupil.  These  data  are 
computed  on  line  ,  allowing  the  mirror  to  track  the  movement  of  the  pupil. 
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ET3  TARGETS  AND  STIMULI 


Hardware: 


our  2D  LASER  SYSTEM  controls 

2  precision  mirror  drives  that  show  a 

repeatability  of  0.04  degree  over  a 

range  of  40  degrees  (can  be  extended  to  80  degrees). 

The  mirrors  move  between  two  points  within  1-2  milliseconds. 

The  high  speed  of  the  mirror  drives  extends  the  range  of  the  laser  stimulus  from  smooth 
pursuit  to  saccade  stimulation. 


A  PC  board  controls 

two  12-bit  D/A  converters  that  provide  the  signals  for  the  mirror  drives. 

A  high  output  frequency  from  the  PC  (up  to  5  KHz)  to  the  D/A  converters  produce  a 
very  smooth  pursuit  target  even  at  high  pursuit  speeds. 

Our  new  laser  system  will  be  equipped  with  a  680  nm  Laser  diode,  that  can  be  switched  by 
one  of  the  digital  lines. 

The  PC  board  also  contains 

16  digital  lines  that  can  drive 

LEDs,  buzzers  or  shutters,  each  line  independently. 

A  set  of  push  buttons  can  be  connected  to  the  eye  tracker  and  to  the  printer  port  of  a 
display  PC.  The  status  of  up  to  for  buttons  is  also  registered. 


Software: 

The  software  that  comes  with  this  hardware  uses  a  text  file  that  contains  wave  forms, 
amplitudes  frequencies,  durations  etc.  that  describe  the  stimulus  in  detail. 

These  text  files  are  transformed  to  a  file  that  can  be  used  by  the  acquisition  program.  All 
stimulus  data  are  being  stored  with  the  eye  movement  data. 

A  lookup  file  contains  the  information  about  which  LED  or  LED  combination  corresponds 
to  which  point  in  space. 

Software  for  PC  display  of  the  calibration  targets  with  push  button  control  Is  already 
available. 

We  currently  prepare  a  program  that  also  displays  on  a  PC  screen  PCX  files  from  a  file  list . 
The  pictures  appear  according  to  the  push  button  input. 

As  the  push  buttons  are  also  connected  to  the  eye  tracker  perfect  synchronisatafon  of 
display  and  data  acquisition  is  maintained. 
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ET3  EYE  MOVEMENT  ANALYSIS 


EYEMAP 

Software  for  Eye  Movement  Analysis 

EYEMAP  is  a  program  for  display  and  analysis  of  eye  movements  especially 
for  analysis  of  saccades  and  smooth  pursuit. 

Using  the  pull  down  menus  with  the  mouse  it  Is  easy  to  move  through  the 
different  features  of  the  program. 

The  mouse  easily  sets  graphical  cursors  in  the  eye  movement  curves  for 
zooming  or  manual  measurements. 

A  context  sensitive  HELP  function  keeps  the  user  informed  about  all  features 
and  about  the  current  task. 

Several  users  of  the  same  program  can  have  different  PERSONAL  SET-UPS  of 
parameters  for : 
graphics, 
analysis, 

paths  and  directories  for  data  and  results  for  fast  and  comfortable 
access  to  the  data. 


Saccades: 

can  be  analysed  automatically,  semi-automatically  and  manually. 

Semi  automatic  evaluation  suggests  a  saccade  and  displays  saccade, 
velocity  and  stimulus  around  the  saccade.  It  allows  the  user  to  correct 
the  points  of  start  and  end  of  each  saccade  and  also  to  correct  the 
stimulus  change  that  belongs  to  a  saccade  if  necessary. 

The  program  calculates: 

start  position,  amplitude,  peak  velocity,  duration,  latency,  peak 
acceleration,  number  of  saccade  after  a  certain  stimulus  change, 
stimulus  amplitude  and  gain,  in  cases  of  smooth  pursuit  it 
calculates  the  stimulus  velocity  and  stimulus  position  at  the  start 
of  the  saccade. 


Smooth  Pursuit: 

The  program  can  calculate  RMS  between  eye  movement  and  stimulus, 
GAIN  of  velocity  amd  position. 

For  good  velocity  gain  the  saccades  can  be  eliminated. 

The  curves  can  be  interpolated  after  the  saccades  have  been  eliminated. 

Several  other  features  are  available:  Differentiating,  Filtering,  Zoom.... 
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ET3  FIXATION  ANALYSIS 

FIXATION: 

This  program  defines  and  analysis  fixations 


CALIBRATION: 

For  reliable  data  analysis  we  use  a  grid  of  calibration  points  on  the  screen.  This  provides 
for  local  linear  calibration  within  each  filed  of  the  grid.  Tilts  or  minor  nonlinearities  are  thus 
being  eliminated.  A  sophisticated  display  shows  tne  result  of  the  calibration  and  helps  to 
select  calibration  points.  The  calibration  allows  to  make  all  further  calculations  in  display 
units  (i.e.  pixels  on  the  screen) 


Fixation  definition: 

Fixations  are  defined  as  valid  data  during  which  the  moved  below  a  certain  speed.  Blinks 
are  detected  separately.  The  speed  threshold  and  the  minimal  duration  of  a  fixation  are 
defined  in  the  setup. 


Analysis: 

INTERVAL  ANALYSIS: 

The  push  button  data  may  be  used  to  define  eye  position  during  a  defined  Interval 
after  the  push  button  was  pressed  over  a  certain  period.  This  feature  may  be  used  to 
check  calibration  and  data  validity.  Delays  and  Interval  length  can  be  determined  in 
the  setup. 


FIXATION  ANALYSIS: 

It  uses  the  saccade/blink  information  to  define  fixations.  Fixations  may  be  "started" 
or  terminated  by  push  buttons  as  well.  Even  a  delay  after  the  button  was  pushed  is 
possible. 

Average  values  for  horizontal  and  vertical  eye  position  are  calculated  together  with 
their  standard  deviation. 

Size  and  other  saccade  parameters  for  the  incoming  and  outgoing  saccades  are 


determined. 

Due  to  the  definition  of  "page"  and  "response"  keys  the  number  of  the  fixation  on  the 
current  page  can  also  be  calculated  as  well  as  the  current  page  number  or  response 
key  number. 

Several  error  codes  define  how  a  fixation  was  terminated  (e.g.  by  blink,  by  push 


button ..)  ... 

All  fixation  are  being  displayed  on  a  colour  screen  an  can  be  accepted  or  rejected 
(i.e.  store  them  anyway,  but  produce  an  error  code).  They  may  also  be  marked  with  a 
number  for  statistical  analysis  of  special  fixations. 


Merging  of  data  (preliminary): 

In  a  second  step  the  results  of  the  fixation  analysis  will  be  merged  with  ’Object’  Information 
i.e.  data  about  the  size  and  position  of  objects  on  the  screen. 
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